The Design of an Auto-Tuning I/O Framework on Cray XT5 System

As high performance computing (HPC) heads towards the exascale era, the computing power surges tremendously and applications will scale to hundreds of thousands cores. Consequently, the amount of data processed and generated will increase dramatically. Nowadays, a parallel shared file system is a must have for a supercomputer. To utilize I/O effectively is essential for an application to scale up. We have developed a mathematical model to describe parallel I/O activities that serves as the basis for an I/O auto-tuning infrastructure for HPC systems. Our current work is in the context of Lustre, but our ideas should be applicable to other distributed file systems. This paper explains our model, which is based on queuing theory, describes the auto-tuning process, and gives experimental results over Lustre on the Cray XT5 that show low relative error. Keywords-I/O; performance modeling; auto-tuning; queuing theory; simulation